Document segmentation based on visual gaps

ABSTRACT

A document may be segmented based on a visual model of the document. The visual model is determined according to an amount of visual white space or gaps that are in the document. In one implementation, the visual model is used to identify a hierarchical structure of the document, which may then be used to segment the document.

BACKGROUND

A. Field of the Invention

Concepts described herein relate to search engines and, moreparticularly, to segmenting documents for indexing by a search engine.

B. Description of Related Art

The World Wide Web (“web”) contains a vast amount of information.Locating a desired portion of the information, however, can bechallenging. This problem is compounded because the amount ofinformation on the web and the number of new users inexperienced at websearching are growing rapidly.

Search engines attempt to return hyperlinks to web pages in which a useris interested. Generally, search engines base their determination of theuser's interest on search terms (called a search query) entered by theuser. The goal of the search engine is to provide links to high quality,relevant results (e.g., web pages) to the user based on the searchquery. Typically, the search engine accomplishes this by matching theterms in the search query to a corpus of pre-stored web pages. Web pagesthat contain the user's search terms are “hits” and are returned to theuser as links.

In an attempt to increase the relevancy and quality of the web pagesreturned to the user, a search engine may attempt to sort the list ofhits so that the most relevant and/or highest quality pages are at thetop of the list of hits returned to the user. For example, the searchengine may assign a rank or score to each hit, where the score isdesigned to correspond to the relevance and/or importance of the webpage.

Local search engines may attempt to return relevant web pages within aspecific geographic region. One type of document that is particularlyuseful for local search engines are business listings, such as abusiness listing found in a yellow pages directory. When indexing abusiness listing, it may be desirable to associate other informationwith the business listing, such as discussions or reviews of thebusiness that are found in other web pages. For example, a web page mayinclude a list of restaurants in a particular neighborhood and a shortsynopsis or review of each restaurant. It is desirable for the localsearch engine to accurately associate the text corresponding to eachrestaurant with the restaurant. Doing so can, for example, increase thesearch engine's knowledge of the business and thus allow it topotentially provide more relevant results to the user.

SUMMARY

One aspect is directed to a method for segmenting a document. The methodincludes generating a visual model of the document, identifying ahierarchical structure of the document based on the visual model, andsegmenting the document based on the hierarchical structure and on thevisual model of the document.

Another aspect is directed to a method of indexing a document. Themethod includes identifying geographic signals in the document andsegmenting the document into sections that correspond to different onesof the identified geographic signals based on a visual layout of thedocument. The method further includes indexing text in the sections ofthe document as corresponding to business listings associated with thegeographic signals.

Yet another aspect is directed to a device that includes a processor anda computer-readable memory. The memory includes programming instructionsthat when executed by the processor cause it to obtain a document thatincludes geographic signals, segment the document into sections thatcorrespond to different ones of the identified geographic signals basedon a visual layout of the document, and index text in the sections ofthe document as corresponding to business listings associated with thegeographic signals.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated in and constitute apart of this specification, illustrate an embodiment of the inventionand, together with the description, explain the invention. In thedrawings,

FIG. 1 is a diagram of an exemplary document illustrating aspectsconsistent with the invention;

FIG. 2 is an exemplary diagram of a network in which systems and methodsconsistent with the principles of the invention may be implemented;

FIG. 3 is an exemplary diagram of a client or server of FIG. 2 accordingto an implementation consistent with the principles of the invention;

FIG. 4 is a flow chart illustrating exemplary operations consistent withaspects of the invention for segmenting documents having localrelevance;

FIG. 5 is a diagram illustrating a portion of an exemplary document;

FIG. 6 is a diagram conceptually illustrating a visual model of adocument;

FIG. 7 is an exemplary diagram of a visual model illustrating ahierarchical structure for document; and

FIGS. 8 and 9 are exemplary diagrams of user interfaces that may bepresented to a user according to an implementation consistent with theprinciples of the invention.

DETAILED DESCRIPTION

The following detailed description of the invention refers to theaccompanying drawings. The detailed description does not limit theinvention.

Overview

A local search engine is described that returns local documents, such asweb documents and business listings, in response to a local searchquery. When indexing and/or categorizing a document, the search enginemay use information from other documents to help describe the document.For example, a third party restaurant review may be used to augment thesearch engine's ability to retrieve relevant restaurants or return therelevant restaurants in response to a search query relating torestaurants.

Some documents, such as some web documents, may contain descriptiveinformation for a number of different business listings. FIG. 1 is adiagram of an exemplary document containing descriptive information forthree different restaurants, “Café Borrone,” “Carpaccio,” and “LeftBank.” The descriptions of the restaurants are each in the paragraphfollowing the restaurant name. From the point of view of the reader, thedocument's visual structure, such as its use of white space, clearlydistinguishes which description corresponds to which restaurant.Conventional automated techniques for analyzing a document, such as thatin FIG. 1, can have difficulty distinguishing the appropriate text thatcorresponds to each restaurant. These conventional techniques may relyon the underlying document structure, such as, for a hyper-text markuplanguage (HTML) document, the document object model (DOM), to attempt todetermine a hierarchical structure of the document. The underlyingdocument structure, however, does not always correspond to the displayedvisual structure of the document. Accordingly, such techniques can beinadequate.

Consistent with aspects of the invention, a segmentation component mayautomatically segment documents based on the visual layout of thedocument.

Exemplary Network Overview

FIG. 2 is an exemplary diagram of a network 200 in which systems andmethods consistent with the principles of the invention may beimplemented. Network 200 may include clients 210 connected to a server220 via a network 240. Network 240 may include a local area network(LAN), a wide area network (WAN), a telephone network, such as thePublic Switched Telephone Network (PSTN), an intranet, the Internet, ora combination of networks. Two clients 210 and one server 220 have beenillustrated as connected to network 240 for simplicity. In practice,there may be more clients and/or servers. Clients 210 and server 220 mayconnect to network 240 via wired, wireless, or optical connections.

A client 210 may include a device, such as a wireless telephone, apersonal computer, a personal digital assistant (PDA), a lap top, oranother type of computation or communication device, a thread or processrunning on one of these devices, and/or an object executable by one ofthese devices. Server 220 may include a server device that processes,searches, and/or maintains documents. Clients 210 and server 220 mayconnect to network 240 via wired, wireless, or optical connections.

Server 220 may include a search engine 225 usable by clients 210. Searchengine 225 may be a local search engine designed to return documentshaving local relevance to the users. Server 220 may include segmentationcomponent 230. Segmentation component 230 may assist search engine 225in indexing or classifying documents by automatically segmentingdocuments having local relevance into sections that correspond todifferent local regions or addresses.

The local documents processed by search engine 225 may be indexed andstored in a data structure, such as database 235. The documents indatabase 235 may be local in the sense that they are associated with aparticular geographic area—though not necessarily the same geographicarea. A document that relates to a business listing, for example, can beconsidered a local document because it is associated with the particularaddress of the business.

A document, as the term is used herein, is to be broadly interpreted toinclude any machine-readable and machine-storable work product. Adocument may be an e-mail, a business listing, a file, a combination offiles, one or more files with embedded links to other files, a newsgroup posting, etc. In the context of the Internet, a common document isa web page. Web pages often include content and may include embeddedinformation (such as meta information, hyperlinks, etc.) and/or embeddedinstructions (such as Javascript, etc.).

Exemplary Client/Server Architecture

FIG. 3 is an exemplary diagram of a client 210 or server 220, referredto as computing device 300, according to an implementation consistentwith the principles of the invention. Computing device 300 may include abus 310, a processor 320, a main memory 330, a read only memory (ROM)340, a storage device 350, an input device 360, an output device 370,and a communication interface 380. Bus 310 may include a path thatpermits communication among the components of computing device 300.

Processor 320 may include any type of conventional processor,microprocessor, or processing logic that may interpret and executeinstructions. Main memory 330 may include a random access memory (RAM)or another type of dynamic storage device that stores information andinstructions for execution by processor 320. ROM 340 may include aconventional ROM device or another type of static storage device thatstores static information and instructions for use by processor 320.Storage device 350 may include a magnetic and/or optical recordingmedium and its corresponding drive.

Input device 360 may include a conventional mechanism that permits auser to input information to computing device 300, such as a keyboard, amouse, a pen, voice recognition and/or biometric mechanisms, etc. Outputdevice 370 may include a conventional mechanism that outputs informationto the user, including a display, a printer, a speaker, etc.Communication interface 380 may include any transceiver-like mechanismthat enables computing device 300 to communicate with other devicesand/or systems. For example, communication interface 380 may includemechanisms for communicating with another device or system via anetwork, such as network 240.

Server 220, consistent with the principles of the invention, performscertain searching or document retrieval related operations throughsearch engine 225 and/or segmentation component 230. Search engine 225and/or segmentation component 230 may be stored in a computer-readablemedium, such as memory 330. A computer-readable medium may be defined asone or more physical or logical memory devices and/or carrier waves.

The software instructions defining search engine 225 and/or segmentationcomponent 230 may be read into memory 330 from another computer-readablemedium, such as data storage device 350, or from another device viacommunication interface 380. The software instructions contained inmemory 330 cause processor 320 to perform processes that will bedescribed later. Alteratively, hardwired circuitry may be used in placeof or in combination with software instructions to implement processesconsistent with the present invention. Thus, implementations consistentwith the principles of the invention are not limited to any specificcombination of hardware circuitry and software.

Segmentation Component Processing

FIG. 4 is a flow chart illustrating exemplary operations consistent withaspects of the invention for segmenting documents having localrelevance. In general, segmentation component 230 may segment a documentbased on the visual layout of a document.

Segmentation component 230 may identify a candidate document forsegmentation (act 401). A candidate document may be one that isidentified to have one or more geographic signals relating to businesslistings. The geographic signals may include information associated witha location, such as a full or partial address of the location, a full orpartial telephone number, and/or a full or partial name of a businessassociated with the location. The locations of the geographic signalswithin a document may be stored in database 235 by, for example, storinga word or character count that indicates where in the document each ofthe geographic signals is located.

The business listings in the document may be identified (act 402) basedon the geographic signals. For instance, a business listing may beidentified when a geographic signal can be determined to define acomplete address and a business name. In some implementations, yellowpage data or other pre-generated lists of businesses can be used toverify the identified business names/addresses.

FIG. 5 is a diagram illustrating a portion of an exemplary document 500.Exemplary document 500 is a HTML web document reviewing a number ofrestaurants. As shown, document 500 may include a document header 510,category labels 520-1 and 520-2, and individual restaurant reviews 530-1through 530-4. As can be seen, document 500 includes four businesslistings, one of which is associated with each of reviews 530-1 through530-4. When indexing this document, it would be desirable to associateeach of the reviews with its corresponding business listing and not withany of the other business listings in document 500. Additionally, headerinformation, such as document header 510 and category labels 520 caninclude useful descriptive information that may beneficially beassociated with the business listings.

In situations in which document 500 is a web page, document 500 may begenerated using a markup language, such as HTML. The particular HTMLelements and style used to layout different web pages varies greatly.Although HTML is based on a hierarchical document object model (DOM),the hierarchy of the DOM is not necessarily indicative of the visuallayout or visual segmentation of the document.

Segmentation component 230 may generate a visual model of the candidatedocument (act 403). The visual model may be particularly based on visualgaps or separators, such as white space, in the document. In the contextof HTML, for instance, different HTML elements may be assigned variousweights (numerical values) that attempt to quantify the magnitude of thevisual gap introduced into the rendered document. In one implementation,larger weights may indicate larger visual gaps. The weights may bedetermined in a number of ways. The weights may, for instance, bedetermined by subjective analysis of a number of HTML documents for HTMLelements that tend to visually separate the documents. Based on thissubjective analysis weights may be initially assigned and then modified(“tweaked”) until documents are acceptably segmented. Other techniquesfor generating appropriate weights may also be used, such as based onexamination of the behavior or source code of Web browser software orusing a labeled corpus of hand-segmented web pages to automatically setweights through a machine learning process.

As an example of assigned weights, consider the HTML element <hr>(horizontal rule). The <hr> element may introduce a weight of 20 beforeand after the element. As another example, the HTML elements <h1>through <h6> are used to start new headings in which <h1> is the mostprominent and <h6> is the least prominent. The corresponding elements</h1> through </h6> are used to end the headings. The various headingsmay, for example, be assigned weights such as, for <h2>, a weight of 50before and 30 after the element (i.e., <h2> may be assigned a weight of50 and </h2> a weight of 30. This reflects the concept that heading textis more likely to be associated with what comes after it than what comesbefore it.

FIG. 6 is a diagram conceptually illustrating a visual model of document500. Visual model 600 includes a number of textual elements 610-1through 610-7, which correspond to the textual elements (document header510, category label 520-1, reviews 530-1 and 530-2, category label520-2, and reviews 530-3 and 530-4, respectively) shown in FIG. 5.Weights 620-1 through 620-6 are assigned between textual elements 620-1through 620-6. The circles shown in textual elements 610-3, 610-4,610-6, and 610-7 represent the corresponding business listing shown inFIG. 5.

For the exemplary visual model 600, assume that text 610-2 and 610-5 areimplemented as <h2> HTML elements, which correspond to weight values of30 for weights 620-2 and 620-5, and the weight value of 60 (50 from text610-5 and 10 from text 610-4) for weight 620-4. Weight 620-1, having avalue of 90, may be calculated as the sum of the weight associated withtext 610-2 (50) and the weight associated with the document header intext 610-1. Document header text 610-1 may be implemented using, forexample, a number of HTML <br> (break) elements or as an HTML headerelement that contribute(s), for example, a weight value of 40 to thetotal value of weight 620-1. Weights 620-3 and 620-6 may be based on,for example, <br> elements after each of the document reviews 530 (i.e.,text 610-3, 610-4, 610-6, and 610-7).

Returning to FIG. 4, the hierarchical structure of the document may bedetermined based on the visual model (act 404). In one implementation,different weights may define different hierarchical levels, in whichlarger weights define higher levels. The lowest level may be determinedas a minimum weight that divides the textual elements containinggeographic signals.

FIG. 7 is a diagram of visual model 600 illustrating a hierarchicalstructure for document 500. Brackets are used to indicate hierarchicalregions. In this example, weight 620-1 is the largest weight and is nearthe top of the document, and may thus indicate that text 610-1 is thedocument title or header (hierarchical level 710). At the nexthierarchical level (hierarchical level 720), weights 620-1 and 620-4define two divisions of three text sections each. Within thishierarchical level (hierarchical level 730), weights 620-2 and 620-5separate the sections. Finally, weights 620-3 and 620-6 separate textsections 610-3 and 610-4, and text sections 610-6 and 610-7(hierarchical level 740). These text sections include the geographicsignals.

Based on the hierarchical levels determined in act 404, portions of thedocument may be associated with the business listings in the document(act 405). In one implementation, the text associated with the smallesthierarchical level surrounding a business listing may be associated withthat business listing. Additionally, text at higher levels that is notassociated with a different business listing may be associated with thebusiness listing. In the example of FIGS. 5 and 6, for instance, text530-2/610-4 may be associated with the business listing for therestaurant “Noodles & Co.,” as well as with the text “Chinese” and thetitle “Top Restaurants in Fairfax, Va.”

The content determined for each of the business listings may beassociated with the appropriate business listing in database 235 (act406). The business listing and its associated content may be indexed asa single combined document. In alternate implementations, the businesslisting and its corresponding content may be indexed separately butassociated with one another via a special field in the index. In theexamples of FIGS. 5-7, for instance, the business listing for “HunanEatery, 4008 University Drive, Fairfax, Va.,” may be associated withreview text 530-1, category label 520-1, and header 510.

Although the segmentation process described with reference to FIGS. 4-7was described as segmenting a document based on geographic signals thatcorrespond to business listings, the general hierarchical segmentationtechnique could more generally be applied to any type of signal in adocument. For example, instead of using geographic signals thatcorrespond to business listings, images in a document may be used (imagesignals). The segmentation process may then be applied to help determinewhat text is relevant to what image. Alternatively, the segmentationprocess described with reference to acts 403 and 404 may be performed ona document without partitioning the document based on a signal. Theidentified hierarchical segments may then be used to guide classifiersthat identify portions of documents which are more or less relevant tothe document (e.g., navigational boilerplate is usually less relevantthan the central content of a page).

Example Search

FIGS. 8 and 9 are exemplary diagrams of user interfaces that may bepresented to a user according to an implementation consistent with theprinciples of the invention. Assume that the user has accessed aninterface associated with a local search engine, such as search engine225 (FIG. 2). As shown in FIG. 8, the user may enter one or more searchterms of a search query via a search field 810. The user mayadditionally enter a geographical region of interest in search field820. In this example, the user has entered the search query “Chineserestaurants whole wheat noodles” and has indicated that the searchshould be performed in the geographic region corresponding to “Fairfax,Va.”

Search engine 225 may implement the search over the pre-indexed documentcorpus stored in database 235. The index may have been generated usingthe techniques discussed above, such that business listings for Chineserestaurants may have also been associated with additional informationthat helps to more fully categorize and/or describe the businesslistings. In this case, search engine 225 may be able to locate aChinese Restaurant in Fairfax, Va. that serves whole wheat noodles.

As shown in FIG. 9, search engine 225, via a user interface, may presentlocal search results 910. For each of search results 910 (or some subsetof the search results), the user interface may provide addressinformation for the business associated with or mentioning the searchresult, a telephone number for the business, a snippet from a documentassociated with the business, a link to more information associated withthe business, a link to directions to the business, and/or a link to oneor more documents that refer to the business. The user interface mayalso provide a map of the area covered by the search. As shown, thefirst search result 910 includes business name and telephone information915, address information 920, a snippet 930 from a document thatdescribes the business (where the document may or may not be associatedwith the business), a link 940 to the document associated with snippet930, and a link 950 for additional documents that refer to the business.

CONCLUSION

Systems and methods consistent with the principles of the invention maysegment a document based on a visual model of the document. Thesegmented document may be used to associate different portions of thedocument with different items, such as different geographicsignals/business listings.

The foregoing description of preferred embodiments of the presentinvention provides illustration and description, but is not intended tobe exhaustive or to limit the invention to the precise form disclosed.Modifications and variations are possible in light of the aboveteachings or may be acquired from practice of the invention.

For example, while a series of acts has been described with regard toFIG. 4, the order of the acts may be modified in other implementationsconsistent with the principles of the invention. Further, non-dependentacts may be performed in parallel.

Also, exemplary user interfaces have been described with respect toFIGS. 8 and 9. In other implementations consistent with the principlesof the invention, the user interfaces may include more, fewer, ordifferent pieces of information.

Further, certain portions of the invention have been described as an“engine” that performs one or more functions. An engine may includehardware, such as an application specific integrated circuit or a fieldprogrammable gate array, software, or a combination of hardware andsoftware.

It will be apparent to one of ordinary skill in the art that aspects ofthe invention, as described above, may be implemented in many differentforms of software, firmware, and hardware in the implementationsillustrated in the figures. The actual software code or specializedcontrol hardware used to implement aspects consistent with theprinciples of the invention is not limiting of the invention. Thus, theoperation and behavior of the aspects were described without referenceto the specific software code—it being understood that one of ordinaryskill in the art would be able to design software and control hardwareto implement the aspects based on the description herein.

No element, act, or instruction used in the present application shouldbe construed as critical or essential to the invention unless explicitlydescribed as such. Also, as used herein, the article “a” is intended toinclude one or more items. Where only one item is intended, the term“one” or similar language is used. Further, the phrase “based on” isintended to mean “based, at least in part, on” unless explicitly statedotherwise.

1. A method for segmenting a document comprising: generating a visualmodel of the document; identifying a hierarchical structure of thedocument based on the visual model; and segmenting the document based onthe hierarchical structure and the visual model of the document.
 2. Themethod of claim 1, wherein generating the visual model of the documentincludes: assigning values to elements of the document used to controlthe appearance of the document, the values quantifying an amount thatthe elements introduce visual gaps in a displayed version of thedocument.
 3. The method of claim 2, wherein the elements of the documentare hyper-text markup language (HTML) elements.
 4. The method of claim2, wherein identifying the hierarchical structure of the documentincludes: identifying higher hierarchical levels for the document ascorresponding to larger assigned values.
 5. The method of claim 1,further comprising: identifying geographic signals in the document. 6.The method of claim 5, further comprising: assigning values to elementsof the document used to control the display of the document, the valuesquantifying an amount that the elements introduce visual gaps in adisplayed version of the document; and identifying a lowest levelhierarchical structure of the document based on a minimum value thatsurrounds the geographic signals.
 7. The method of claim 5, whereinsegmenting the document further includes: associating sections of thedocument with the geographic signals based on the identifiedhierarchical structure.
 8. The method of claim 7, wherein segmenting thedocument further includes: associating sections of text surrounding thegeographic signals with the geographic signals.
 9. The method of claim7, wherein segmenting the document further includes: associating textthat is at a higher hierarchical level than the geographic signals andthat does not itself include a geographic signal with the geographicsignals as header text.
 10. A computer-readable medium containingprogramming instructions for execution by a processor, thecomputer-readable medium comprising: programming instructions forgenerating a visual model of a document that includes at least onegeographical signal; programming instructions for identifying ahierarchical structure of the document based on the visual model; andprogramming instructions for associating the at least one geographicalsignal with portions of the document based on the identifiedhierarchical structure of the document.
 11. A method of indexing adocument comprising: identifying geographic signals in the document;segmenting the document into a plurality of sections that correspond todifferent ones of the identified geographic signals based on a visuallayout of the document; and indexing text in the plurality of sectionsof the document as corresponding to business information associated withthe geographic signals.
 12. The method of claim 11, wherein one of thegeographic signals includes a postal address of a business.
 13. Themethod of claim 11, wherein segmenting the document includes: generatinga visual model of the document; and segmenting the document based on thevisual model.
 14. The method of claim 13, wherein segmenting thedocument includes: identifying a hierarchical structure of the documentbased on the visual model.
 15. The method of claim 13, whereinsegmenting the document includes: assigning values to elements of thedocument used to control the appearance of the document, the valuesquantifying an amount of visual gaps that the elements introduce in adisplayed version of the document.
 16. The method of claim 15, whereinthe elements of the document are hyper-text markup language (HTML)elements.
 17. The method of claim 14, wherein identifying thehierarchical structure of the document includes: assigning values toelements of the document used to control the display of the document,the values quantifying an amount that the elements introduce visual gapsin a displayed version of the document; and identifying higherhierarchical levels of the document as corresponding to larger assignedvalues.
 18. The method of claim 14, wherein segmenting the documentfurther includes: associating sections of text surrounding thegeographic signals with the geographic signals.
 19. The method of claim14, wherein segmenting the document further includes: associating textthat is at a higher hierarchical level than a geographic signal and thatdoes not itself include the geographic signal with the geographic signalas header text.
 20. A system comprising: means for identifyinggeographic signals in a document; means for segmenting the document intoa plurality of sections that correspond to different ones of theidentified geographic signals based on a visual layout of the document;means for indexing text in the plurality of sections of the document ascorresponding to business listings associated with the geographicsignals; and means for storing the indexed text.
 21. The system of claim20, further comprising: means for performing a local search based on asearch query and the indexed text.
 22. The system of claim 21, whereinresults of the local search include a business listing relevant to thesearch query, the business listing being displayed with a snippet from adocument relevant to the business listing.
 23. A device comprising: aprocessor; and a computer-readable memory coupled to the processor andcontaining instructions that when executed by the processor cause theprocessor to: identifying a document that includes signals, segment thedocument into a plurality of sections that correspond to different onesof the identified signals based on a visual layout of the document, andindex text in the plurality of sections of the document as correspondingto the signals.
 24. The device of claim 23, wherein the signals includegeographic signals and the text is indexed to correspond to businesslistings associated with the geographic signals.
 25. A methodcomprising: identifying a document for indexing into a search engineindex; identifying references to business listings in the document;generating a model of the document based on a visual layout of adisplayed version of the document; hierarchically segmenting thedocument based on the model and the references to the business listings;and associating content in the document with particular ones of thebusiness listings based on the hierarchical segmentation of thedocument.
 26. The method of claim 25, wherein the document is ahyper-text markup language (HTML) document, and wherein generating themodel further comprises: assigning values to HTML elements of thedocument based on a predetermined decision of how much white space theHTML elements contribute to the visual layout of the displayed versionof the document.